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ABSTRACT 

To determine the relative importance of school and 
individual factors in the determination of science learning, 
achievement test scores were analyzed in relation to individual- and 
school-level factors in a national sample of about 2,000 tenth-grade 
students participating in the Longitudinal Study of American Youth. 
Hierarchical linear analyses showed that individual measures 
accounted for most of the variance. Previous achievement was the 
preponderant influence on subsequent achievement. Nonetheless, 
initial science attitude, instructional time, the home environment, 
and exposure to mass media were also significant individual-level 
influences on science achievement. (Author) 
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ABSTRACT 

To determine the relative importance of school and individual factors in the 
determination of science learning, achievement test scores were analyzed in 
relation to individual- and school-level factors in a national sample of about 
2,000 tenth-grade students participating in the Longitudinal Study of American 
Youth. Hierarchical linear analyses showed that individual measures accounted 
for most of the variance. Previous achievement was the preponderant influence 
on subsequent achievement Nonetheless, initial science attitude, instructional 
time, the home environment, and exposure to mass media were also significant 
individual-level influences on science achievement. 



A theory of educational productivity (Walberg, 1981) holds that three groups of nine factors a) 
aptitude consisting of 1) ability or prior achievement, 2) motivation, and 3) age or developmental 
level; b) instructional 3) time and 4) quality; and c) the psychological environments of the 6) 
classroom, 7) home, 8) peer group, and 9) mass media are the major and consistent determinants of 
* a ^ n al outcomes. The theory has guided the compilation of more than 120 research syntheses 
of 8,000 comparisons in small-scale experimental and correlational studies (Fraser, Walberg, 
Welch, & Hattie, 1987) and 23 regression analyses of achievement obtained from mostly national 
surveys of about 250,000 students in sue subjects of primary and secondary school study (Paschal & 
Stariha, 1992). Although 303 (or 89%) of 341 regression weights were in the theoretically- 
predicted direction, much of data were cross-sectional; and the analyses made little use of structural 
modelling that takes into account indirect causation, reverse causality, measurement error, and lack 
of optimal scaling of variables. 

To improve upon earlier estimates, Reynolds and Walberg (1991) employed the LISREL 7 program 
(Joreskog & Sorbom, 1988) which incorporates such structural modeling features. They tested the 
consistency of the findings with national longitudinal data on junior and senior high school students 
in mathematics and science over a one-year time span. When adjusted for the specification, 
measurement, and scaling errors in the data, the coefficients were generally higher than indicated 
by previous regression analyses and were generally in the expected direction. Therefore, it could 
be concluded that the model held up well under more rigorous tests than had been previously been 
made. This study, which included cross-validation, corroborated the mediating effects of the 
productivity factors not evident in previous studies. 

However, one potential criticism of both regression and structural analyses is that they fail to take 
into consideration the multilevel nature of educational data which typically includes school 
classroom and student levels (Burstein, 1980; Walberg, 1984; Bryk & Raudenbush, 1992). 
Psychologists can argue that educational factors are individual level measures and corresponding 
individual-level analyses should prevail in both experimental and correlational research. 
Statisticians and sociologists, on the other hand, point out that the teacher and students influence 
one another, and therefore their interdependence violates the statistical assumptions of the analysis 
and claims too many independent degrees of freedom, that is, corresponding to the number of 
students rather than the number of schools. An early and unsatisfactory solution for this problem is 
to cany out analyses at both individual and aggregate levels and look for robust findings across 
specifications (Walberg & Ariderson, 1968). 
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To answer such statistical criticism it has recently become possible to analyze multiple levels For 
what has been termed "hierarchical linear modeling" analyses (HLM, Bryk & Raudenbush 1992) 
several programs are now available that allow simultaneous analysis of two (and even three) levels 
HLM programs employ a modified regression approach to estimate the multi-level coefficients and 
their significance. They also provide estimates of the variance accounted for by each level much 
* C . ?o?™ S10 , n P r ° vides at a single level. Like regression, however, HLM lacks the special features 
ct LI5REL 7 such as multivariate coefficients in the case of multiple dependent variables and 
correction of attenuation attributable to measurement error and non-linear relations. Thus it 
appears worthwhile subjecting well collected data sets to both structural and hierarchical tests for 
the magnitude and significance of effects. 

Methodology 

Sample Design 

This study involved 2535 tenth-grade public school students and 51 schools that were a part of the 
Longitudinal Study of American Youth (LSAY; Miller, Suchner, Hoffer, & Brown, 1990) The 
sample design is two-stage stratified by region (East, West, North, and South) and by urbanity 
Crural, suburban, and urban). Missing data on science achievement and other variables reduced the 
sample sizes somewhat as indicated in a subsequent section. 

Date collection for this study took place in the fall of 1987, spring of 1988, and fall of 1988 While 
students were tested for science achievement in the first collection in the fall of 1987 usine NAEP 
tests (National Assessment of Educational Progress, 1986), attitudes towards science, motivation 
peer environment, and weekly homework, the second collection in the spring of 1988 involved 
student and teacher surveys. In addition, parents were interviewed regaining student attendance 
home resources, and exposure to mass media (television and books). Teachers provided 
formation such as class environment and instructional quality. The third data collection & the fall 
fLu3 reP S?^ e c ° Uectl0 1 n ? f achievement and attitude date from the students. This study 
tocused on the grade 10 cohort and the data collected while in grades 10 (1987) and 1 1 (1988). 

Analytic Method 

The longitudinal Study of American Youth dataset consisted of students residing in school and 

?enTh a T A H lth0U K gh rt Re ^ 10ldS (1991 ' 1992 > have rc P° rted ftom A™ sevemh and 

t^e hiSf °^ t , USmg / trU 5 tUral m ^ tM> S, the methodology for this investigation incorporated 
tiie hierarchical nature of students and schools in a procedure called hierarchical linear moddUnfr 
iwwn ^ i e ?. u ^ tlonal Ksvuch about students residing in classrooms located within schools 
within school districts, within states, etc. often fails to account adequately for the nouDine 
attributes. The hierarchical order with each group influencing the members of me £o?p iffiglrt 
and behaviour can lead to problems with homogeneity of students within groups. If the school 
effects are ignored when comparing students, then the problem of biased significance tests will lead 
of Slf^ and . Mfere " CeS (R?udenbush & Bryk, 1986; Raudenbffi 1988) . The^S 
of unexplained variance m student performance must first be partitioned into the school anditodent 
level components if this bias is to be avoided. In previous studies (Young, 1990, 1991a 1991b in 
press; Young & Eraser, 1992a, 1992b), the school effect ranged from 9 to 19 percent of fte totS 
unexplained variance in physics achievement, depending upon the age of the snident If not 
removed from the statistical analysis, this effect could have led to the underestimation of me 
^riEif™ -r SUltlng in ,£ e rejCCti0n 0f ^ nul1 hypothesis. For examp e the fmding of 
^nr tS- 51 ^^ 1 ^ X tf*"™* c ° uM ^ attributable to the underestimation of the stendard 
error It is impcranye, therefore, that the total amount of variance is estimated usinc a multilevel 

ffiffn *$, empl ° yCd ** USC ° f £f Hier ^^al Linear Model (HLH) develop^ 

^^^i^^^JSSS^ 2 ' developed by Brykl Raudenbush> Se£ 

K!?. 5 ? 001 ^ ° n stud ? nt Performance are multilevel in nature, standard regressions may 
be misleading, and often underestimate the effects of the school and overestimate student 
characteristics such as sex differences. The multilevel model consists of a sepamteKeen schoo 
regression equation for each b coefficient in the regression model. The IherLwcalLS Mode 
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(Raudenbush, 1988; Raudenbush & Bryk, 1986) was used in this study to investigate the effect of 
the classroom environment, instructional quality and time. The multilevel analysis was conducted 
using science achievement as the student outcome variable, in an attempt to explain these student 
differences in science achievement 



Table 1 

Description of Student Level and School Level Variables 



Outcome Variable: 

Science achievement (science) - NAEP cognitive subtests 

Science knowledge (scikn 11) 1 8-20 items 

Science uses (sciusl 1) 17. ig items 

Science integration (scintl 1) 18-19 items 



Student Level Variables: 
Sex 

Science attitude (attitude) 
Interest 
Usefulness 



Prior science achievement (ability) - NAEP cognitive subtests 
Science knowledge (scikn 10) 22 items 

Science uses (sciuslO) 19 items 

Science integration (scintlO) 22 items 

Motivation (motiv) - academic motivation 
Persistence 
Instrinsic motivation 

Instructional time (time) - material covered 
Cuts class now and then 
Parent reported homework 
Student reported homework 

Home environment (home) 
Parent education 
Parent expectations 
Duncan socioeconomic index 

Peer environment (peer) - most of my friends 
Plan to go to college 
Are really good students 
Do well in science 

Mass media (media) - out of school reading 
Read six or more books 
Read a newspaper often 



Male= 1, Female = 0 

3 item composite 

4 item composite 



sum of 2 items about trying hard at school 
sum of 3 items about putting off study 

1 = yes, 0 = no 
hours per week 
hours per week 

5 = advanced degree, 1 = high school or less 
5 = advanced degree, 1 = high school or less 
continuous scale 

1 = yes, 0 = no 
1 = yes, 0 = no 
1 = yes, 0 = no 

1 = yes, 0 = no 
1 = yes, 0 = no 



School Level Variables 

Instructional quality (quality) - teacher reported instructional focus 

Lab technique skills 4 = nea vy emphasis, 1 = none 

Conducting experiments 4 = greater than weekly, 1 = very rarely 

Written reports required 4 = greater than weekly, 1 = very rarely 

Class environment (class) - teacher reported class attributes 
Percent of students to get bachelor's degree 
Percent of students to take more science than required 



Note: All items and scales were standardized before combining into composites 
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Table 2 

Descriptive Statistics for Student and School Level Variables 



Variable 



N 



Mean 



Standard 
Deviation 



Minimum 



Maximum 



Student Level 
Science 
Sex 

Attitude 
Ability 
Motivation 
Time 
Home 
Peer 
Media 



2003 


.00 


2003 


.49 


1850 


.00 


1964 


.03 


1981 


.02 


2001 


.01 


1988 


.01 


2003 


.04 


1855 


.02 



School Level 

Science 

Urban 

Ability 

Attitude 

Motivation 

Time 

Home 

Peer 

Media 

Qual 

Class 



47 
47 
47 
47 
47 
47 
47 
47 
47 
47 
47 



-.02 
2.13 
-.01 
.04 
.00 
-.03 
-.04 
.01 
.01 
.06 
.05 



.90 


-2.49 


2.20 


.50 


.00 


1.00 


.89 


-2.45 


2.03 


.88 


-1.68 


2.64 


.85 


-3.71 


1.65 


.69 


-1.69 


2.93 


.77 


-2.00 


1.82 


.73 


-1.23 


.93 


.76 


-.83 


1.24 



.28 


-.69 


.57 


.77 


1.00 


3.00 


.33 


-.78 


. .83 


.17 


-.35 


.44 


.23 


-.47 


.85 


.19 


-.45 


.53 


.28 


-.70 


.79 


.18 


-.31 


.64 


.21 


-.65 


.72 


.53 


-1.45 


1.33 


.63 


-1.00 


1.38 



Variables used in this Study 

The Longitudinal Study of American Youth included science achievement test items developed by 
il Assessment of Educational Progress (NAEP, 1986), and self-reported student 
„?Sl^. .^T' $tU , dent and teacher survevs ^d parent interviews provided 
SdZS" VStT attendance home resources, and exposure to mass media such as television 
SSi&^'n Tea ^f rs P rov . lded , information about the classroom environment and instructional 
ShTn'intV^^ 16 - ? nS1StCd °? 3 combination ° f items, except for sex of the student Before 
combining the items into composite scales, the items were standardized so that they contributed 
equally towards the combined scale. The years 10 and 1 1 science achievement measures, SlO 
f^Lf- tn^' 7 • m ? d sc * nll >X iusll > a«i sdnill, were previously standardized and then 
S lS h h° SL " gl H- SCaleS> a - bUlty a ? ds «'« n «. respectively. These variables are described in 
Table 1 , and the descriptive statistics provided in Table 2. 

Preliminary Analyses 

The initial stage in HLM analysis involved the estimation of the total variance of the deoendent 

XnY^"? acl "T ment ^ t0taI variance was ^ decomposed mtoSeen 

^ ^^"'^dent variance as reported in the following sections, in order to determmethe 
source of variations in science achievement. The student level independent variables e^mined 

SKte »Tf Tf dS <P« 115 attitUde) ' P rfor scie ^ ce achievement <S?«S 

Z„ % J ? ■ im0tlV \' uls ? uctlonal time (class and homework), home environment (tome) 
peer characteristics (peer) and mass media (media). School effects investigated inemded 
insmicuonal quality (science laboratory emphasis) ml classroom env£,St 
student aggregates were also analysed at the school level (level 2) in order to identify anv 

SSS? 6 1 t° *5? effCCtS ° f *» h0mC environment on student ShKaSS 

vary from school to school or are the effects similar across all schools. 
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For each of the indicators, there were between 1855 and 2003 students and 47 vchmu m ,ii„M« r 

£S^t2SS5f scicnce home and clioom ? ~^SS 

When initial hierarchical linear model analyses were run on science achievement the within 
t S ht°±JTf a t!l Ce W ? S a8 L W i\ Ue between schools variance was TWs mean" 

S133 £ £ ZlH C ?h?e %) W2S 31 ^ StU , dent leveK lev^I i'albles were 

EES 

Table 4 

Variance Components Analysis 



Effect 



Within School 
Variance 



Between School 
Variance 



No independent Variables 

8 Student Variables (Level 1) 

8 Student & 10 School Variables 
(Levels 1 8c2) 

5 Student & 2 School Variables 
(Levels 1 & 2) 



.8105 

.5070 (37.4%) 
.5068 

.5073 



.04554 

.01134 (75%) 
.00779 (82.9%) 

.00638 (86.0%) 



The Hierarchical Linear Model 

achievement was specified for the Level-1 and LcvelTmodels: ^ S ° m 

Science^ = + rjj 

P0j - Y00 + "0j Equation 2 

where Science^ represents the dependent variable, science achievement for student i in school j 
P 0 j represents the intercept or mean science achievement for all students in j schools and r ' 
represents the Level-1 error term normally distributed with a mean of zero and a variance of «2 v U 
represents the grand mean of science achievement for student/ S^j S^SSSl 
associated with school j (set at a mean of zero and a variance of x^. 

!" V ™ C ° mp0nentS ' nodel was ^ysed, ^e total variance was 0.8560 with <fi being 
2 ( °\ -"V 00 ^ 0,04554 (5% > as described ^ Table 4. That is 95 percent of the 
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to 0 ^sSTiTl^T W 10 m0dd ' &eK was a reduction » ° 2 from 0.8105 

£S ^ 3 ™ Pereent.reductoon). The predictors were sex of the student, attitude toward 
envSnSSf SC1Cn< L e a <^y™t (ability), motivation of the student, insSS Si? hSme 
ShcET"*- 1 *? ^a™™" and mass media (see equation 3). A 1 of these prefix w£e 
statistically significant with the exception of sex, motivation and t >er environment SSSte 5? 
The weak and insignificant sex slope (p,. = -0.06, t = -1.73) indicated Uiat, whUe £ corrclaSi 
slightly with science achievement in a positive direction (that is. boys onSS S K?„~ 

J^S5*? , t ,hc ^ was a si - gnificant reduction in ^^oSSS^^t^SSS^ 

K^* 11 leav,n .g a native s ope (that is, girls appeared to outpcrfoim boys£ hX e ° W oX 
the sex differences m science achievement were accounted for by student cKtS«£f 
attitude towards science and prior ability. These two Slopes were St^S^rZ^ ■* 

effSKoS TllSt Va ?T' J* ^"^owSscience SngT^ ^Sve 
criect ip 2j - 0.05, t - 2.39) than pnor science ability (p 3 . = 0.53, t = 26.19). The stronc nositive 
P" 0 /^" 0 ^ ability slope was the most significant student level predictor. Smilari - fiXnal 
SiSS 8 nUm ^ 1 clas ^. and hours spent doini homeworkXhome eTvbonmen 

and mass media were significant in explaining science achievement (p 5i = 0. 1 t = 4 00 T = 0 09 

ol «V a „ : H and hi = 0M> ' = 3 ; 2 ° although weaker. StudSt motivation £ = 0 00 t = 

teStt = ° - 02, ' = t? 4) ^ " 0t appCar t0 C ° ntribute ^"Mtly towards 
explaining student differences in science achievement These variables were later removed fi-mr, 
the final explanatory model of science achievement. _ removed from 

Tables 

Hierarchical Linear Model results for Science Achievement- 
Student Level Variables Model 



Effect Coefficient 
Student Level Intercept .q Q2 



t-ratio 



Sex 
Altitude 



-0.81 

-0.06 -1.73 
0.05 2.39* 



mS?** °' 53 2619* 

Motivation om Q lg 

Tirne 0.11 4 00* 



Home 0 ' 09 
Peer 
Media 



3.73* 



0.02 0.74 
0.08 3.20* 



Note: * statistically significant at p a 0.05 



u.£.. u *i*uji. aip~ u.uj 1 

s.gma squared = 0.5070 (within school variance) and tou = 0.01134 (between school variance) 

Sden 3 = Tinl + S + fcjA? ^ + PsjAbilityjj + p^Motivationn Equations 
+ P 5j Time ij + p 6j Home u + p^Peer u + p 8j Mediai j + r j j J 4 

P0j - 700 + TpiUrbanj + ^Ability; + ^Attitude; + ^Motivation; 
+ TCsHcmej + ^Peetj + ^Tvlediaj + ^gTime; + ^dasa 
+ 7ioQuahty + u 0j Equanon4 
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Table 6 

Hierarchical Linear Model results for Science Achievement: 
Initial Explanatory Model 



Effect Coefficient 



t-ratio 



Student Level Intercept n on n *a 

School Level Variables °* 04 



Urban 

Ability q.24 

Attitude -o!ll 

Motivation -O.OO 

Home 0 !o8 
Peer 

Media -6.06 
Time 
Class 



-0.01 -0.23 
2.11* 
-0.69 
-0.04 
0.64 

0.00 .0.01 
0.44 

-0.25 -1.49 



^ -0.14 -3 09* 

Quality o 06 i no 

Student Level Variables 



Sex 

Attitude 5.05 
Ab^ty o.52 
Motivation q 01 

Time 
Home 
Peer 
Media 



0.06 -1.65 
2.51* 
25.18* 
0.33 

0.11 417* 
0.09 3 67* 

0.02 0.81 
0.07 3.02* 



Note: * statistically significant at p = 0.05 



sigma squared - 0.5068 (within school variance) and tau = 0.00779 (tetwecn school variance) 
P0j = W0 + Y0iAbility j + Y02 Class j + u 0j Equadon6 
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Tabic 7 

Hierarchical Linear Model results for Science Achievement- 
Final Explanatory Model 



Effe * Coefficient 



t-ratio 



Student Level Intercept QOl n « 

School Level Variables ' * 

Ability 0.19 



2 35* 

0.13 -3!21* 



Class 

Student Level Variables 

Attitude 0< o5 

£ bmt y 0.52 ^ JD 

l mx * 0.12 4.90* 

Home o.lO 4 14* 

Media 0 0 8 3 25* 



2.32* 
25.55* 



Note: * statistically significant at p = 0.05 " " 

sigma squared = 0.5073 (within school variance) and tau = 0.00638 (between school variance) 

Discussion 
Conclusions 

%22JSSS TosfnS^" P roduct i vit y * largely driven by individual-level 



*2 



and complicated than measured in this study. Process of models educational productivity verify 
such direct, indirect, and mediated influences. Moreover, effects may be reciprocal or may depend 
on particular constellations of family, school, and individual factors. 
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